[00:14.640 --> 00:18.240]  And welcome back, everybody, to DEF CON 28 Safe Mode.
[00:18.360 --> 00:22.940]  Continuing on with the Blue Team Village OpenSOC CTF walkthrough,
[00:22.940 --> 00:24.920]  we're going to be talking about Moloch today,
[00:24.920 --> 00:28.880]  and we have Bashar Shama here to give us a quick discussion
[00:28.880 --> 00:30.520]  and a walkthrough on the tool.
[00:30.740 --> 00:32.960]  Welcome, Bashar.
[00:33.320 --> 00:35.380]  Thank you. Hello, everyone.
[00:35.380 --> 00:40.200]  I am Bashar, and I'm going to go over how we're going to use Moloch tomorrow
[00:40.200 --> 00:41.780]  during the CTF.
[00:41.780 --> 00:45.780]  I was actually introduced to Moloch two years ago
[00:45.780 --> 00:49.020]  playing the same exact CTF that you're going to play tomorrow.
[00:49.020 --> 00:52.620]  I played it two years ago, and I really, really fell in love with Moloch.
[00:52.620 --> 00:56.020]  So since then, I started playing with it,
[00:56.020 --> 00:58.060]  tinkering with it, just use it as much as I can,
[00:58.060 --> 01:00.220]  and I keep using it day to day.
[01:01.740 --> 01:07.520]  The goal today is to really prepare you on how to use Moloch on the CTF tomorrow.
[01:07.760 --> 01:11.260]  So we will have time for questions at the end.
[01:11.260 --> 01:17.220]  Please post any questions that you have to the TechWorkshops Track 1 channel in the Discord.
[01:17.620 --> 01:23.600]  The moderators will monitor your questions, and we're going to ask them at the end,
[01:23.600 --> 01:24.640]  towards the end of the session.
[01:24.640 --> 01:27.240]  So keep your questions thrown away.
[01:27.240 --> 01:29.880]  We'll have some time at the end to go over some questions.
[01:31.000 --> 01:33.960]  So let's go ahead and get started.
[01:34.960 --> 01:39.240]  So a very brief intro on what Moloch is.
[01:39.240 --> 01:41.860]  It's pretty much a free open source tool.
[01:42.320 --> 01:50.080]  It's really a network analysis tool that you can use to analyze a large volume of packet data, or PCAPs.
[01:51.040 --> 01:55.300]  The simplest way to think about it is, if you've ever used Wireshark,
[01:55.300 --> 02:00.520]  it's like a Wireshark frontend with a huge database backend,
[02:00.520 --> 02:04.940]  so you can search tons and tons of PCAP data, gigs of data, hundreds of gigs of data,
[02:04.940 --> 02:08.240]  of PCAPs very easily, very quickly.
[02:09.280 --> 02:16.240]  That's a very, very, very 10,000, over 2,000 foot overview of Moloch.
[02:16.480 --> 02:21.520]  For our purpose here, and for the CTF tomorrow, what I have done is, actually,
[02:21.520 --> 02:28.620]  I have Moloch set up, and I got some, about four gigs of data as a sample
[02:28.620 --> 02:32.540]  from the network forensics training of March 2015.
[02:32.620 --> 02:37.240]  I downloaded these PCAPs, and I pretty much loaded them into Moloch
[02:37.240 --> 02:42.640]  to kind of show you how we can use this tool to do our investigation
[02:42.640 --> 02:46.100]  and answer the questions during the CTF tomorrow.
[02:46.860 --> 02:53.340]  So, our scenario for today is, you're part of the security team,
[02:53.340 --> 03:00.100]  and you receive a call saying, hey, around 1 p.m. today, or not today,
[03:00.100 --> 03:04.980]  1 p.m. on March 12, 2015, our main company site has been de-fixed.
[03:04.980 --> 03:08.240]  Somebody has changed the way our website looks like.
[03:08.300 --> 03:15.000]  All we know is, we have this image of a file showing as a frog on our main site,
[03:15.000 --> 03:18.460]  and we don't know how this happened. Can you please help us?
[03:18.680 --> 03:22.360]  And all you know is, you have access to packet capture.
[03:22.480 --> 03:25.300]  You have packet capture, and you have access to Moloch.
[03:25.300 --> 03:27.920]  So, let's walk right through it.
[03:28.360 --> 03:32.900]  So, if you have never seen Moloch before, this is what it looks like.
[03:32.900 --> 03:37.360]  This is when you land or you log in.
[03:37.360 --> 03:39.600]  You will land pretty much in the sessions tab,
[03:39.600 --> 03:42.900]  and that's where you're going to probably spend most of your time tomorrow,
[03:42.900 --> 03:44.040]  in the sessions tab.
[03:45.260 --> 03:50.380]  Usually, I would like to go and just drop down here on the time, on the date,
[03:50.380 --> 03:54.260]  and I would do all time, all day, just to have an understanding
[03:54.260 --> 03:56.320]  of how big the incident is.
[03:56.980 --> 04:00.600]  For the purpose of tomorrow, again, it's going to be a very specific time period.
[04:00.600 --> 04:06.560]  But this is where I would start, and let's go ahead and deep dive into the investigation
[04:06.560 --> 04:08.260]  and how can we use Moloch.
[04:08.260 --> 04:13.840]  So, the first piece of information we know is the date of the incident.
[04:14.040 --> 04:18.520]  So, we know it's on 2015.
[04:18.520 --> 04:22.160]  So, as you can see, Moloch can give us this ability to just click through
[04:23.260 --> 04:27.180]  and decide on the dates that we want to investigate.
[04:27.240 --> 04:28.960]  I'm going to do 12.
[04:28.960 --> 04:32.100]  So, we have our date here. That's going to be our starting date.
[04:32.320 --> 04:36.940]  And then I'm going to just copy it, paste it here, and I'm going to change this to 13,
[04:37.440 --> 04:42.100]  which will show us here now, oh, you're looking at one day time range.
[04:42.320 --> 04:45.540]  So, I have my time range set up. I'm going to go search.
[04:46.560 --> 04:49.680]  And now I'm going to narrow it down to that 24-hour period
[04:50.620 --> 04:53.440]  to kind of figure out what happened during that day.
[04:53.440 --> 04:59.440]  Now, the other piece of information that we were given was it was our main company's website.
[04:59.780 --> 05:06.140]  So, how can I search for that? Probably I want to look for our host name.
[05:06.140 --> 05:11.600]  So, what I can do is I can type host, and then Moloch would automatically parse
[05:11.600 --> 05:17.800]  the different fields based on the protocols that exist in the packet.
[05:17.800 --> 05:24.380]  So, if it was HTTP traffic, then Moloch will say, okay, well, this is an HTTP host name.
[05:24.380 --> 05:27.500]  If it's an email, then it's an email host name and so on.
[05:27.880 --> 05:30.960]  In my case, because I want to see everything, how can I post it?
[05:30.960 --> 05:35.700]  I can just say host. And to specify what I'm looking for in Moloch,
[05:35.700 --> 05:38.480]  I would just say equal equal.
[05:39.400 --> 05:47.060]  So, equal equal means show me everything that matches exactly our host name.
[05:47.680 --> 05:53.980]  On ad.ase. Okay? That's the first bit of it. And I can run that.
[05:54.200 --> 05:59.800]  I also know, since it's our website, I want to say, well, it's a website,
[05:59.800 --> 06:01.720]  it's going to only run on two ports, for example.
[06:01.720 --> 06:07.280]  So, to kind of add more queries into Moloch, what you need to do is,
[06:07.280 --> 06:11.580]  you just do two amp signs as an end.
[06:11.740 --> 06:15.540]  And I'm going to open parentheses and add the ports that we need.
[06:15.540 --> 06:20.600]  So, port equals 80, just like we did before.
[06:22.120 --> 06:30.200]  And port equals 443, which we know they both are HTTP ports.
[06:30.200 --> 06:34.020]  Now, before we go any further, you know, under ports I can say, well,
[06:34.020 --> 06:39.320]  I can specify I want it to be a destination port, I can have it be a source port, or so on.
[06:39.540 --> 06:43.860]  For, just to keep it general, and we can see everything here, I'm just going to say, okay,
[06:43.860 --> 06:48.320]  these ports, this host name, and let's see what we find.
[06:48.560 --> 06:51.000]  And I messed up something, of course.
[06:51.180 --> 06:55.140]  Oh, I'm saying end here, and that should be or.
[06:55.140 --> 06:58.300]  So, it's either port 80 or port 443.
[06:59.440 --> 07:06.920]  So, in that case, I just do two pipelines, and run that.
[07:07.940 --> 07:09.460]  Not pipelines, but pipes.
[07:09.460 --> 07:14.720]  And now I'm saying, okay, show me everything for this host on port 80 and port 443.
[07:14.720 --> 07:17.140]  And now I can see the traffic.
[07:18.300 --> 07:22.140]  So, let's go and dive deeper.
[07:22.960 --> 07:31.840]  On this traffic, I can see all kind of requests, and let's just open something very randomly.
[07:31.940 --> 07:36.500]  Let's open up this request, and I see it's a get request.
[07:36.500 --> 07:39.380]  And before we go down, let's spend some time here.
[07:39.380 --> 07:44.260]  So, idcmalloc will parse all the fields in the packet.
[07:44.260 --> 07:50.280]  So, I can click on any of these, and I can say, okay, well, what protocol did it come from?
[07:50.280 --> 07:52.220]  Which IP, which ports?
[07:53.260 --> 08:00.360]  If it's HTTP packet, then it will also parse the method, the status code, all of these things.
[08:00.680 --> 08:02.580]  It will also parse the user agent.
[08:02.580 --> 08:11.980]  Let's say I'm interested in knowing all the user agents that happened in that specific day that visited our website.
[08:12.040 --> 08:18.640]  I can easily click on user agents, and I can say, okay, export unique user agents with counts.
[08:18.640 --> 08:29.800]  Once I click that, I will see a new page showing me the unique user agents, along with how many times we've seen this in that specific day, and the other one.
[08:29.800 --> 08:34.780]  So, we only see two different user agents, nothing abnormal, nothing suspicious.
[08:34.780 --> 08:37.580]  So, nothing to worry about here.
[08:38.300 --> 08:45.360]  If we scroll a bit down, we can actually see the actual requests, the raw requests.
[08:45.360 --> 08:50.600]  So, I can see which hosts they're requesting, where the URI they're going after, and so on.
[08:51.320 --> 08:56.800]  Because I know we had an incident, and I know the website has been defaced,
[08:56.800 --> 09:01.440]  the attacker must have sent some kind of data to our website.
[09:01.440 --> 09:04.540]  So, most likely they will not be doing a GET request.
[09:04.540 --> 09:07.280]  When we see GET, it's just pulling data from our website.
[09:07.280 --> 09:10.040]  They're going to be posting or sending some kind of data.
[09:10.040 --> 09:12.660]  So, this excludes this.
[09:12.860 --> 09:18.160]  So, I'm going to click GET, and we don't want GET anymore, so I'm just going to say net GET.
[09:18.420 --> 09:20.940]  That automatically adds that to our query.
[09:22.580 --> 09:24.820]  And let's hit search and see what we find.
[09:24.820 --> 09:28.280]  So, now we went down to only 12 entries.
[09:28.280 --> 09:34.500]  Okay, that's much easier to kind of go through and investigate.
[09:35.080 --> 09:42.360]  Again, we're not sure what happened, but one thing I can sort for is by time,
[09:42.360 --> 09:47.460]  to kind of understand the timeline of these different requests and the events.
[09:48.220 --> 09:51.000]  And this just opens on the random.
[09:53.680 --> 09:55.800]  So, again, same kind of thing.
[09:56.040 --> 09:59.660]  So, now it's a POST request.
[10:02.920 --> 10:05.400]  And it's just a request in index.php.
[10:10.470 --> 10:12.650]  And I see a test on sleep.
[10:12.650 --> 10:14.850]  I'm not sure what all this stuff really means.
[10:14.850 --> 10:18.770]  And I also see this jumper stuff. What is this?
[10:19.070 --> 10:23.250]  So, if we look at the header, which I've already displayed for you,
[10:23.250 --> 10:25.510]  it turns out it's actually encoded with gzip.
[10:25.510 --> 10:30.710]  Gzip is just a method of compression that web browsers use to compress the data
[10:30.710 --> 10:33.170]  to transfer the least amount of traffic.
[10:33.370 --> 10:37.550]  So, the nice thing with Molek, what we can do is I can just click uncompress,
[10:38.210 --> 10:41.750]  and now it automatically will decode this packet for us.
[10:41.750 --> 10:45.550]  And now I can easily read it and see, oh, okay, this is what loaded,
[10:45.550 --> 10:47.190]  this is what the page is showing.
[10:47.570 --> 10:52.110]  And from what we see here, nothing of interest yet.
[10:52.110 --> 10:54.130]  Okay, that was useful.
[10:54.130 --> 10:57.730]  But let's keep going. Let's find out what happened.
[10:58.270 --> 11:02.330]  So, another request, and see this.
[11:02.330 --> 11:05.410]  But this is like, this looks like an IP address.
[11:05.410 --> 11:07.470]  But what is all this? I don't know.
[11:07.930 --> 11:13.810]  Well, what we can do is Molek has Cybershift built in.
[11:13.950 --> 11:17.290]  Well, Cybershift is a separate open source project,
[11:17.290 --> 11:22.010]  that you can actually just go and do Cybershift,
[11:22.010 --> 11:25.510]  and you can load it outside of...
[11:26.510 --> 11:28.330]  Cybershift, there we go.
[11:31.600 --> 11:35.540]  And you can just load it outside of Molek, and you can do your decoding.
[11:35.540 --> 11:36.380]  But let's get back here.
[11:36.380 --> 11:39.480]  The beauty of this is when we do it on Molek,
[11:39.480 --> 11:43.860]  it will automatically take that packet data and put it in for us.
[11:43.880 --> 11:45.640]  So, what is Cybershift?
[11:45.640 --> 11:47.880]  It's pretty much a tool, a web GUI,
[11:47.880 --> 11:52.780]  that you can use to do different decoding of different languages,
[11:53.440 --> 11:55.360]  decoding methods, and so on.
[11:55.360 --> 12:00.600]  So, in this case, it's taking the hex code and just decoding it.
[12:00.800 --> 12:05.560]  And while I'm looking at this, I can see these percentage signs.
[12:05.960 --> 12:09.400]  Percentage sign means it's a URL kind of thing.
[12:09.400 --> 12:12.860]  So, I can easily just drag the URL decode.
[12:12.860 --> 12:16.700]  And now I can see the decoded message here.
[12:16.920 --> 12:21.480]  Now I can see, oh, it's trying to ping this IP address,
[12:21.480 --> 12:25.000]  which is the same IP that's trying to visit.
[12:25.000 --> 12:27.200]  Okay, this is interesting.
[12:27.200 --> 12:30.140]  I'm not sure if they were actually able to ping,
[12:30.140 --> 12:35.120]  but it looks like this IP is trying to do something here.
[12:37.380 --> 12:40.540]  Let's keep going, see what else can we find.
[12:42.860 --> 12:46.620]  Next packet, same thing.
[12:46.620 --> 12:50.060]  Tests, okay, nothing here.
[12:50.480 --> 12:54.180]  Next packet, so on.
[12:55.640 --> 12:57.780]  Oh, there's an NC.
[12:57.840 --> 12:59.460]  Okay, what is this? Hold on a second.
[12:59.460 --> 13:02.220]  Let's pull it up again in CyberChef.
[13:02.220 --> 13:05.860]  Do the same method, decode the URL.
[13:05.940 --> 13:10.960]  And now we see an NC IP address and port.
[13:10.960 --> 13:15.040]  So, NC stands for Netcat, which is the utility attackers can use
[13:15.040 --> 13:19.040]  to have a server connect back to them and get a show in that box.
[13:19.040 --> 13:24.920]  So, what this is saying is, okay, connect back to my IP over this port.
[13:25.600 --> 13:28.980]  So, what this means, if this actually succeeded,
[13:28.980 --> 13:34.520]  that means our server connected back on this IP address.
[13:34.800 --> 13:37.820]  Well, let's see if this actually happens.
[13:37.820 --> 13:42.280]  I'm going to take that port, I'm going to clean all this up,
[13:42.700 --> 13:46.600]  and then I'm going to add the IP address.
[13:47.780 --> 13:49.500]  It's not going to be the source.
[13:52.140 --> 13:57.400]  And port equals the port that we're looking for.
[13:57.480 --> 13:58.520]  Let's search for it.
[13:58.520 --> 14:00.420]  And we have traffic. Uh-oh.
[14:00.880 --> 14:02.420]  This is not good.
[14:02.420 --> 14:06.020]  Looks like our box, which is the source here,
[14:06.020 --> 14:08.800]  connected back to the server over this port.
[14:09.040 --> 14:14.240]  We can see how many packets and the amount of data that was sent back and forth.
[14:14.600 --> 14:18.900]  So, immediately, I would say, okay, well, this is the first connection
[14:18.900 --> 14:22.460]  that has the highest number of packets, that might have something interesting in it,
[14:22.460 --> 14:24.960]  so let me open it up.
[14:26.540 --> 14:33.280]  Scroll down, and we can actually see the whole conversation back and forth now.
[14:33.280 --> 14:37.920]  Well, it looks like the attacker ran a command, which is id,
[14:37.920 --> 14:40.620]  which is equal to whoami on Windows,
[14:40.620 --> 14:45.400]  to figure out who the attacker is running as on this box,
[14:45.400 --> 14:47.320]  and they're running as apache.
[14:47.580 --> 14:50.300]  Then they try to figure out which folder they're in,
[14:50.300 --> 14:55.220]  and they try to access file systems.
[14:55.220 --> 14:56.820]  Okay, this is not good.
[14:57.140 --> 15:02.460]  And then they did this cat-index.php, which looks like it's our website,
[15:02.460 --> 15:07.740]  so they're looking into what's inside our main file and so on.
[15:07.900 --> 15:09.840]  Okay, what is all this?
[15:09.840 --> 15:12.420]  Oh, and then I see another netcat command here,
[15:13.440 --> 15:18.920]  and saving the connection as a cm0.php file,
[15:18.920 --> 15:22.220]  and our box, locally.
[15:22.500 --> 15:26.820]  So I'm interested in knowing what this is, but let's keep going.
[15:26.820 --> 15:29.020]  Let's see what else they do in this box.
[15:30.180 --> 15:32.760]  And if we come here, they actually did it for us.
[15:32.760 --> 15:35.140]  They did a cat on this file that they created.
[15:36.420 --> 15:40.220]  Okay, when they did that, we can see, oh, it's a PHP backdoor.
[15:40.480 --> 15:44.420]  This is not good, so they used a PHP backdoor on our site.
[15:45.060 --> 15:48.800]  So how am I going to now figure out what did they do?
[15:49.060 --> 15:51.840]  I know because of how backdoors work,
[15:51.840 --> 15:57.820]  they have to visit this PHP file explicitly to kind of load the command they want to run.
[15:57.820 --> 16:03.240]  So what I can say to Molek is, well, show me all the URIs.
[16:03.520 --> 16:06.060]  So when we say a URI is...
[16:07.880 --> 16:12.380]  pretty much everything after the domain will be a URI,
[16:12.380 --> 16:14.300]  so anything from here on, that's a URI.
[16:14.300 --> 16:18.480]  So I want to say, show me everything that has this in it.
[16:18.600 --> 16:20.980]  And since I can't just do, like, all,
[16:20.980 --> 16:25.120]  what I do in Molek is I just use wildcards at the beginning,
[16:25.120 --> 16:29.360]  so I don't care what's before this, and the end, meaning I don't care what's after it,
[16:29.360 --> 16:33.320]  just show me anything that has this string in the URI.
[16:33.580 --> 16:36.620]  And when I search it,
[16:37.680 --> 16:42.060]  now, here, we can see all these URLs or URIs
[16:42.060 --> 16:44.820]  with all these different links in them.
[16:44.820 --> 16:46.400]  What are they trying to do?
[16:46.600 --> 16:51.240]  Well, how can I know, like, how can I get all this in one nice place?
[16:51.240 --> 16:53.540]  I can just click on info,
[16:53.540 --> 16:57.240]  and I want to say export unique URI with two counts.
[16:58.380 --> 17:00.320]  Now, Molek will take this for us and tell us,
[17:00.320 --> 17:03.980]  okay, well, this command was run five times,
[17:03.980 --> 17:06.560]  this command was run three times, and so on.
[17:06.560 --> 17:11.000]  So now I kind of know which commands they ran,
[17:11.000 --> 17:13.320]  because I can see it's command equal cat,
[17:13.320 --> 17:16.900]  command equal cat, or else, or whatever they're trying to do.
[17:17.180 --> 17:20.280]  And I kind of see here there's a JPEG file as well.
[17:20.280 --> 17:26.080]  Okay, so this is how they probably got that JPEG file
[17:26.080 --> 17:29.460]  or the image of the fraud into our website.
[17:29.820 --> 17:35.920]  So let's find out where did they, like, how did they do this?
[17:37.240 --> 17:43.000]  Let's go back here, and let's do...
[17:43.000 --> 17:47.700]  Instead of this, we are looking for that specific JPEG file.
[17:48.220 --> 17:53.020]  So I'm going to do wildcard and show me anything that has JPEG in it.
[17:53.960 --> 17:55.180]  Load it up.
[17:56.060 --> 17:58.180]  And it's going to show me a lot of stuff.
[17:59.620 --> 18:03.200]  I'm going to say probably it's going to be the biggest file,
[18:03.200 --> 18:06.640]  so I'm going to sort by the size of the data.
[18:08.060 --> 18:10.740]  And this is the biggest file that we have.
[18:14.970 --> 18:16.350]  So it's loading.
[18:17.010 --> 18:19.950]  And here I can see, yeah, it's an image file.
[18:23.290 --> 18:25.510]  But I'm not sure what it looks like.
[18:25.510 --> 18:30.090]  So what I can do in Moloch is I can just click on show images and files,
[18:30.090 --> 18:33.310]  and it will actually render it for us right here in the browser.
[18:34.390 --> 18:37.370]  Now, let's say, you know, I'm not comfortable with doing this
[18:37.370 --> 18:41.890]  or I want to dig in deeper to it or I want to understand what actually happens
[18:41.890 --> 18:45.030]  using Wireshark because I've never used Moloch before.
[18:45.030 --> 18:50.210]  At any session, at any point, you can always just click on download PCAP,
[18:50.210 --> 18:53.090]  and it will kind of, like, save the PCAPs for you.
[18:53.470 --> 18:59.090]  So you can open up in Wireshark and, you know, do your analysis manually
[18:59.090 --> 19:00.710]  if you'd like to.
[19:04.650 --> 19:08.950]  Pretty much that's how they get the file,
[19:08.950 --> 19:12.270]  and that's how they deface our website using a backdoor.
[19:12.270 --> 19:17.810]  If we have time, I think we have another... I think that's pretty much it.
[19:17.810 --> 19:23.870]  Okay, so with that being said, I think that's the highlight of Moloch
[19:23.870 --> 19:27.290]  and how to use it. I want to keep some time for questions.
[19:28.590 --> 19:30.950]  Please let me know any questions you've got.
[19:31.170 --> 19:35.070]  I'm also going to be in Discord. I'll be on Twitter if you need anything else,
[19:35.070 --> 19:38.450]  but I'm going to be here waiting for any questions that you guys have.
[19:39.610 --> 19:40.610]  Cool.
[19:41.290 --> 19:42.470]  Thank you, Bashar.
[19:42.630 --> 19:47.990]  I went ahead and put a note in the text window under the workshop track one
[19:47.990 --> 19:54.370]  with a link over to Recon Episec's OpenSOC Moloch Discord channel.
[19:54.830 --> 19:56.250]  So definitely check that out.
[19:56.250 --> 19:59.570]  Obviously, hit Bashar up on Twitter and Discord,
[19:59.570 --> 20:03.890]  but we're trying to help everybody kind of connect with the right people here
[20:03.890 --> 20:05.370]  so you can get the help you need.
[20:06.870 --> 20:07.590]  Perfect.
[20:08.830 --> 20:10.990]  Thanks a lot, Bashar. I appreciate it.
[20:11.130 --> 20:13.570]  Thank you. Thank you. I appreciate your time. Thank you.
[20:13.590 --> 20:14.210]  Take care.
